O Routledge 

8 Taylor & Francis Group 


Journal of Educational and Psychological Consultation , 25:90-108, 2015 
Copyright © Taylor & Francis Group, LLC 
ISSN: 1047-4412 print/1532-768X online 
DOI: 10.1080/10474412.2014.929961 


School-Wide Educator Evaluation for 
Improving School Capacity and Student 
Achievement in High-Poverty Schools: Year 1 
of the School System Improvement Project 


LINDA A. REDDY and RYAN J. KETTLER 

Rutgers University 

ALEXANDER KURZ 

Arizona State University 


Tloe Un ited States is in an era of high-stakes evaluation of educa¬ 
tors (i.e., teachers and principals), the results of which are used to 
inform human capital decision making (i.e., recruitment , hiring, 
retention, and dismissal), which in turn impacts school capacity 
and student learn ing. The present article describes the School Sys¬ 
tem Improvement (SSI) Project, a school-wide educator evaluation 
initiative that includes 22 high-poverty schools. Hoe primary goal 
is to present the Year 1 planning for implementation of educator 
evaluation, highlighting the use of system consultation for plan¬ 
ning. Tloe SSI Project team includes researchers in school psychol¬ 
ogy and special education, school administrators, and teachers. 
Tloe project aims to implement a multimethod educator evaluation 
system that generates scores for informing targeted , evidence-based 
professional development. Project goals are to increase educator 
competencies that lead to improved student achievement. Finally, 
directions for training and recommendations for future school 
reform are presented. 


Because the Nation at Risk Report (U.S. Department of Education, 1983), con¬ 
cerns over American students’ educational progress and learning outcomes 
have been at the forefront of the national dialog on education. More recently, 
the passage of the No Child Left Behind Act of 2001 mandated that all 
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children were to reach proficiency status on state assessments of educational 
performance by 2014. NCLB requires proficiency testing in reading and 
mathematics for all students from third grade through eighth grade, and 
at one grade level in high school. The legislation has brought increased 
attention to many educational challenges facing the nation, including but 
not limited to student achievement gaps, standardized testing, school system 
accountability, and teacher quality. 

Despite the ambitious goals stated in this legislation, our nation’s educa¬ 
tional system remains at risk. In a recent report the United States was ranked 
17th in the world for education (Economist Intelligence Unit, 2012). Countries 
such as Latvia, Chile, and Brazil have demonstrated academic gains at a rate 
three times faster than American students (Hanushek, Peterson, & Woess- 
mann, 2012). In 2009, the United States was ranked 25th out of 34 countries 
participating in comprehensive mathematics assessments administered by 
the Organization for Economic Cooperation and Development Programme 
for International Student Achievement (Fleischman, Hopstock, Pelczar, & 
Shelley, 2010). Likewise, The Nation’s Report Card, an annual report from the 
Institute for Education Sciences regarding annual student proficiency levels 
on the National Assessment of Educational Progress (NAEP), indicates our 
nations’ continuing struggle to meet NCLB goals. For example, in 2009, 38% 
of fourth graders and 30% of eighth graders scored at or above proficient in 
reading. In the same year, 38% of fourth graders and 32% of eighth graders 
scored at or above proficient in mathematics (Rampey, Dion, & Donahue, 
2009). 

In light of these findings, much controversy remains about whether 
NCLB has been effective in improving our nation’s schools, as well as whether 
meeting the goal of all children being proficient by 2014 is realistic. Recog¬ 
nizing these questions and the limitations surrounding NCLB, Congress has 
granted states a mechanism to apply for and receive waivers from meeting 
its requirements. These waivers were granted in exchange for agreeing to 
raise standards, improve accountability, and improve teacher effectiveness. 
As President Obama stated, these waivers “will make for a fairer system and 
one that focuses on what matters most: getting the whole school system 
to perform better in terms of student learning” (CNN Wire Staff, 2012). 
Recognizing the urgent need for new models of school-wide reform, in 2012 
the U.S. Department of Education announced the need for improved human 
capital management systems (HCMS) in schools. An HCMS is designed to 
focus on the restructuring of school systems’ organizational effectiveness by 
bringing the processes responsible for promoting effective teaching (i.e., re¬ 
cruitment, induction, evaluation, professional development) under a unified 
umbrella. Centered on rigorous educator evaluation, the application of HCMS 
principles to school districts can potentially improve instructional practices, 
educator effectiveness, and ultimately student learning (e.g., Odden & Kelly, 
2008; Odden, 2011). 
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Bipartisan support for this movement is evident in the recent appro¬ 
priation of federal funding ($285 million) for the 2012 U.S. Department of 
Education Teacher Incentive Fund (TIF) program. “This year’s TIF competi¬ 
tion will be a catalyst for local leaders and educators ready to pursue a new 
and improved vision for school leadership and teaching” (U.S. Department 
of Education, 2012). The TIF program encourages participants to establish 
school-wide systematic reform for improving HCMS through rigorous edu¬ 
cator evaluation linked to targeted professional development (PD), career 
ladder opportunities, and incentives for educator effectiveness. 

The primary goal of this article is to provide an introduction to the 
School System Improvement (SSI) Project, a TIF funded school-wide re¬ 
form project focused on educator evaluation and job-embedded PD in high- 
poverty schools, with a focus on Year 1 planning for implementation of 
an educator evaluation system (EES). The collaborative process used by 
researchers, school administrators and teachers to plan for implementation 
is presented. This article accomplishes three additional objectives. First, the 
core principles and role of school psychology in educator evaluation re¬ 
form are outlined. Second, the use of system consultation in implementation 
planning for Year 1 is described. Finally, recommendations for future school 
reform are presented. 


SSI PROJECT 

The SSI Project was a group application among four school districts (22 
schools) and two universities. As required by the TIF program, all school 
partners met the definition of high poverty, with 50% of enrolled students or 
more receiving free or reduced lunch. The primary purpose of TIF funded 
projects is to develop and implement a comprehensive integrated HCMS 
that includes strategies to recruit, hire, develop, retain, and reward effective 
educators. Within the HCMS, principal and teacher evaluation systems and 
PD are implemented. 

For purposes of the SSI Project, 22 high-poverty schools in New Jersey 
comprised of approximately 60 school administrators, 1,110 teachers, and 
10,000 students began a partnership with researchers from Rutgers, The State 
University of New Jersey and Arizona State University to build a rigorous 
and innovative EES for teachers and principals that recognizes and rewards 
effective teaching and leadership practices over the course of a 5-year period. 
The SSI Project team aligned the EES with the New Jersey Educator Effec¬ 
tiveness Task Force (2011) recommendations and the Excellent Educators for 
New Jersey (EE4NJ) requirements, concurrent mandates from the New Jersey 
Department of Education (NJDOE). During Year 1, NJDOE mandated that all 
schools adopt a new educator evaluation system. The SSI Project offered the 
four school districts significant support for this statewide mandate. 
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The SSI Project team designed the EES using an approach that evalu¬ 
ates educator effectiveness based on (a) observational data from measures 
such as the Danielson Framework for Teaching (DFT; Danielson, 2013) 
and the Classroom Strategies scale observer form (CSS; Reddy & Dudek, 
2014; Reddy, Fabiano, Dudek, & Hsu, 2013); (b) student growth data from 
assessments such as the New Jersey Assessment of Skills and Knowledge 
and the Measures of Academic Progress (MAP® Northwest Evaluation Asso¬ 
ciation; 2013); and (c) teacher self-report measures such as the Instructional 
Learning Opportunities Guidance System (MyiLOGS; Kurz & Elliott, 2012; 
Kurz, Elliott, Kettler, & Yel, 2014) and the CSS-Teacher Form (Reddy & 
Dudek, 2014; Reddy, Dudek, Fabiano, & Peters, in press). The EES gener¬ 
ates scores that inform four performance levels of effectiveness. The EES 
also provides data to inform empirically supported PD for teachers and 
principals. That is, the SSI Project will use the collected data on inputs, 
processes, and outcomes to customize PD according to educators’ individual 
profiles. Funding through TIF further supports these PD efforts by enabling 
capacity building at the district level via one-on-one coaches for teachers 
and principals, thus establishing a personalized, job-embedded approach to 
PD. The PD will be hierarchically implemented through SSI Project leader¬ 
ship principals and teachers and 51 school-based master mentor teachers 
identified as effective by the EES and hired by districts with SSI Project 
funds during Year 2. Rutgers-based leadership principals and teachers will 
implement group PD meetings with principals and teachers at each of the 22 
participating schools during Year 2. During Year 3, the Rutgers leadership 
principals and teachers will train and supervise the 51 school-based master 
mentor teachers in delivering group and individualized PD to teachers in the 
four partner districts. Individualized PD will be delivered in Project Years 3 
through 5. 

Consistent with TIF program requirements, the school districts of the SSI 
Project will reward teachers and principals for EES scores that fall in the ef¬ 
fective and highly effective performance levels providing performance-based 
compensation in the form of stipends. The SSI Project does not include merit 
pay systems or changes in base salary structures for teachers or principals that 
alter school district operating budgets. That is, stipend funds come directly 
from the U.S. Department of Education TIF program. 

For Year 1, the primary goal was to plan for implementation of the EES. 
This goal was accomplished by meeting with school district administrators to 
discuss the alignment of the SSI Project Goals and EES with current statewide 
teacher and principal evaluation policy mandates and school district- specific 
instructional improvement plans. Plans for communicating SSI Project goals 
and EES components to all school personnel, as well as training school 
personnel (principals and teachers) on EES measures were accomplished. 
Next, we describe the core principles and role of school psychology in 
educator evaluation. 
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THE CORE PRINCIPLES AND ROLE OF SCHOOL 
PSYCHOLOGY IN THE SSI PROJECT 

Within a multidisciplinary team, school psychologists can lead or contribute 
to the educator evaluation movement. The SSI Project team applies core 
principles of serving individuals with special needs at the system level for 
schools and districts. Subsequently, many of the core components of the 
Model for Comprehensive and Integrated School Psychological Services (the 
NASP Model; National Association of School Psychologists, 2010) are ap¬ 
parent in the design of the SSI Project’s EES and PD. The NASP Model 
specifies many aspects of the role of school psychologist, subdivided into 
the following categories: (a) practices that permeate all aspects of service 
delivery; (b) direct and indirect services for children, families, and schools; 
and (c) foundations of school psychologists’ service delivery. While the NASP 
Model specifies multiple practices, services, and foundations relevant to the 
practice of school psychology, one example of each is particularly relevant 
to the SSI Project and will be discussed in this section. 

Data-based decision making and accountability. Data-based decision 
making and accountability is a school psychology practice that affects all 
aspects of service delivery and is a core competency for the SSI Project team. 
According to the NASP Model , “School psychologists have knowledge of 
varied models and methods of assessment and data collection methods for 
identifying strengths and needs, developing effective services and programs, 
and measuring progress and outcomes” (NASP, 2010, p. 4). The EES is 
designed to transform educator evaluation that has historically been static and 
one-dimensional into a dynamic, multifaceted process that integrates data 
from tests, observations, and informant-report to provide results that are fair 
and balanced. These results are connected to a continuum of individualized 
and ongoing PD services designed to improve the effectiveness of principals 
and teachers. The EES also provides ongoing progress- monitoring data on 
the effectiveness of the PD. Examples of data-based decision making and 
accountability cited in the NASP Model include a foundation of data from 
multiple sources, use of valid and reliable assessment techniques, selection 
and implementation of evidenced-based services, and use of information 
and technology to enhance decision making. The leaders of the SSI Project 
collaborate with school district administrators, directors of curriculum, and 
teacher representative committees to review and select assessment tools and 
PD services that have substantial evidence supporting their use in schools. 
A key component of the project is to collect evaluation data on these as¬ 
sessment tools and services, once they have been implemented, to be sure 
that they are working as intended within the SSI Project’s school districts. 
All of these processes are supported by the SSI Project online portal that 
is designed to organize and analyze data from a diverse set of measures 
applied to evaluate teachers and principals and inform PD in four districts. 
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School-wide practices to promote learning. According to the NASP 
Model, “School psychologists have knowledge of school and systems struc¬ 
ture, organization, and theory; general and special education; technology 
resources; and evidence-based school practices that promote learning and 
mental health” (NASP, 2010, p. 6). The SSI Project team and school partners 
discussed the importance of the HCMS (system-level change) for each school 
by measuring the effectiveness of instruction —the universal preventive 
intervention—and connecting that knowledge seamlessly within an iterative 
cycle of PD and monitoring. To enhance HCMS, the SSI Project embraced 
many assumptions outlined in Deming’s (1993) Total Quality Control (TQC) 
for school reform (i.e., reducing fear, engaging all in self-improvement, 
collaborative leadership, continuous job training and supports, focus on 
student achievement) with the exception of assumptions related to elimi¬ 
nating standards of effectiveness. These assumptions were used as guiding 
principles for the SSI Project and are briefly discussed in the consultation 
entiy and contracting phases. School psychologists who have knowledge 
and skills in system consultation, system theory, and TQC principals can 
lead or contribute to effective change in HCMS. 

While school psychologists have typically focused on the strengths and 
weaknesses of individual students, those roles are changing as the field 
becomes more preventive and moves toward response to intervention mod¬ 
els. To foster school-wide collaborative efforts among general and special 
educators, the SSI Project features an EES that includes measures relevant to 
all stakeholders. For example, the EES allows general educators to report (via 
the CSS-Teacher form) on the implementation of specific behavior manage¬ 
ment strategies that provide school psychologists critical data for evaluating 
Tier 1 prevention. The EES further allows special educators to input and track 
(via MyiLOGS) student objectives and monitor instructional differentiation 
between the overall class and individual students. As such, the SSI Project 
is designed to create the conditions for collaborative, data-based decision 
making (Years 1-2), which are enhanced by using site- and project-based 
instructional coaches (Years 3-5). 

The principles of assessment for intervention are as applicable at a 
systems level as they are at the individual level. Examples of school-wide 
practices to promote learning cited within the NASP Model include the 
incorporation of evidence-based strategies for effective practices, support 
of a respectful atmosphere using principles of systems theory, promotion 
of learning environments that feature engagement and academic growth, 
and application of the problem-solving process at the building level. The 
SSI Project team and school partners discussed the importance of school 
system evidence-based practices, respectful communications to and from 
stakeholders (e.g., teachers, principals, district administrative leaders), and 
contributions that persons from each role offer to the SSI Project and school 
reform efforts. The EES includes measures of student engagement and aca- 
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demic growth, among a number of related constructs, to allow for diagnostic 
solving of any problems at the system level. 

Research and program evaluation. Research and program evaluation 
is a foundation of school psychological service delivery. According to the 
NASP Model, “School psychologists have knowledge of research design, 
statistics, measurement, varied data collection and analysis techniques, and 
program evaluation sufficient for understanding research and interpreting 
data in applied settings” (NASP, 2010, p. 8). As a true collaboration between 
school districts and researchers, the SSI Project provides schools access to 
assessment and intervention techniques that are supported by the most cur¬ 
rent research. The SSI Project employs a team of statistical and measurement 
specialists, dedicated to collecting, combining, and interpreting data from 
multiple sources to provide information that is the most useful for schools. 
Longitudinal change in these data, over the 5-year life of the project, will 
then be used in an evaluative process that closes the loop of evidence- 
based practices. Examples of research and program evaluation cited within 
the NASP model include use of empirical foundations for effective practices, 
support for classroom teachers in progress monitoring, incorporation of 
data collection techniques in the evaluation of services, and employment 
of technology to obtain that data. The constituent pieces of the EES and 
PD are all empirically founded and teachers are supported by SSI Project 
personnel in Years 3 through 5 to monitor progress and improve practice. 
The EES involves multiple and varied sources of data, entered and analyzed 
within a contemporary online portal that is easy for educators to use. 

Collectively, data-based decision making, school-wide practices, and 
research and program evaluation are important foundational principles for 
the SSI Project. Next, the system consultation model used for the SSI Project 
Year 1 is described. 


SSI PROJECT YEAR 1: 

GOALS AND SYSTEM CONSULTATION MODEL 

The SSI Project goals for Year 1 were to (a) discuss the alignment of SSI 
Project goals and the EES with current statewide policy mandates, (b) un¬ 
derstand districts’ instructional improvement goals and plans, (c) learn the 
unique organizational and cultural structure of each school system, (d) com¬ 
municate the SSI Project goals and EES with all school personnel and further 
secure investment, (e) establish EES training for Year 1, and (f) implement 
monitoring plans for Year 2. Goal attainment was achieved in part through 
the ongoing solicitation and review of stakeholders’ input (via meetings, 
telephone or Skype calls, and e-mail correspondence) in relation to statewide 
policy and local union collective bargaining agreements. System-level con¬ 
sultation was used in part for achieving Year 1 goals. 
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The SSI Project team engaged in consultation grounded in systems the¬ 
ory and principles of organizational functioning (Beer & Spector, 1993; Beer, 
1980) in Year 1. The SSI Project team used a modified version of Kurpius, 
Fuqua and Rozecki’s (1993) organizational development consultation model, 
as well as principles of innovation diffusion (Rogers, 2003) and implementa¬ 
tion science (Fixsen, Blase, Duda, Naoom, & Van Dyke, 2010; Fixsen, Naoom, 
Blase, Friedman, & Wallace, 2005) as a framework for working with schools 
as organizations. Kurpius’s seven-stage consultation model includes (1) pre¬ 
entry; (2) entry; (3) contracting; (4) problem exploration; (5) information 
gathering, problem confirmation, and goal setting; (6) solution searching and 
intervention selection; and (7) evaluation and termination. For Year 1, the 
four stages (pre-entry, entiy, contracting, information gathering) of Kurpius’s 
seven-stage consultation model were employed in accordance with the Year 
1 primary goal of planning for EES implementation, and are described in 
this section. Principles of innovation diffusion and implementation science 
guided planning in Year 1. Examples of principles of innovation diffusion 
included the SSI Project team’s examination of school districts’ commit¬ 
ment to fully adopt the proposed EES, perceived advantage of the EES 
in comparison to their current evaluation system, alignment of the EES in 
relation to district goals and state mandates, and understanding and use 
of data generated by the EES (additional methods used by the SSI Project 
team to enhance perceived advantage and EES adoption are outlined in the 
recommendation section). Likewise, the literature on implementation science 
offered a valuable foundation for planning. The SSI Project Team used 
strategies such as detailing the goals, focus, and delivery of EES training and 
conceptualizing the measurement approach of EES implementation fidelity 
(see Table 1 for details). 

The SSI Project consultation model is process-focused, ongoing, collab¬ 
orative, iterative, and top-down. Due to the infrastructure of school district 
decision making, we engaged in a process whereby we initially collaborated 
with district administrative leaders (e.g., superintendents, assistant superin- 


TABLE 1 Multidimensional Measurement of School-Wide Implementation Fidelity 

Measurement targets 

Definitions 

General model use 

Specific model delivery 
Performance feedback of 
model delivery 

Progress monitoring of 
model delivery 

School personnel 
responsiveness 

Adaptation 

School-wide adoption and implementation 

Implementer delivery and quality of delivery 

Implementer feedback on delivery and quality of delivery 

Progress in implementer delivery following performance 
feedback 

Comprehension and use of model components; Social 
validation of model delivery 

Implementer modifications to model delivery to enhance 
contextual fit 
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tendents, directors of curriculum) on the SSI Project goals, EES, and align¬ 
ment with statewide evaluation policy mandates and school district- specific 
instructional improvement plans. During these meetings information was 
obtained on the unique school district organizational structures, as well as 
roles and responsibilities of administrators, principal and teacher evaluators, 
and union collective bargaining agreements. Permissions were then obtained 
to proceed with our consultation process with building-level principals, vice 
principals, and teachers. A top-down process was adopted due to the high- 
stakes nature of educator evaluation and changes in statewide policies dur¬ 
ing Year 1. Collaborators were district administrative leaders (e.g., super¬ 
intendents, assistant superintendents), school administrative leaders (e.g., 
principals, vice-principals), teachers, curriculum supervisors, and auxiliary 
educational faculty that comprise District Educator Assessment Committees 
(DEACs), and School Improvement Panels (SIPs). 

Consultation was guided by the belief that schools are organizations 
that include complex and interconnected inputs, processes, and outcomes 
that involve adults, students, and local and state legislative bodies (e.g., local 
unions, statewide policies). A goal of the school organization is to socialize 
staff in a manner that enhances the likelihood that school organizational 
goals will be achieved. This goal is often accomplished through a variety 
of human capital management processes that include recruitment, hiring, 
orientation (i.e., induction), development, supervision, and incentives (e.g., 
tenure and promotion). Thus, human capital management processes are 
critical for improving principal leadership, teacher effectiveness, and student 
growth in achievement; improvement in these three areas is the goal of the 
SSI Project. Aligned with Deming’s TQC, the SSI Project’s core principles 
are rigorous evaluation linked to ongoing PD and supports, improvement 
for all, collaborative and effective leadership, and strong focus on student 
achievement. 

Prior to Year 1 (pre-entry, first 3 months), the SSI Project team accom¬ 
plished several key tasks to inform planning. First, a statewide assessment 
of school districts that qualified as high poverty under federal regulation 
was conducted. The initial list was then cross-checked with the NJDOE 
roster of priority schools (i.e., schools mandated to receive state monitoring 
and supports). Second, schools that qualified as high poverty (i.e., 50% or 
more students enrolled received free or reduced lunch) and that were not 
identified as priority schools were contacted to discuss the project. Third, 
meetings were scheduled with district administrative leaders interested in 
partnering with the SSI Project team. During these initial meetings, the SSI 
Project team described the grant priorities and project goals in relation to 
state policy and detailed how the project would benefit the school district 
and personnel. Likewise, the SSI Project team collected initial information 
on the school organizational structure, culture, core values, and needs. A 
critical goal of these initial meetings was for the SSI Project team to forge 
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strong relationships with school district administrative leaders. Relationship 
development was accomplished in part by recognizing district leaders’ and 
the SSI Project team’s unique and shared competencies and mutual inter¬ 
ests in improving school capacity for student success (Erchul, Grissom, & 
Getty, 2009). The SSI Project team informally assessed the cultural context 
of each school district. While all SSI Project schools were identified as high 
poverty and included a significant percentage of minority students, the SSI 
Project team discovered that each school district had its unique infrastruc¬ 
ture, perceptions of support and PD, and mechanisms for communicating 
between and among school personnel. In some cases the communication 
processes among school administrators, instructional coaches, and teachers 
were very limited, resulting in the identification of barriers and options 
(i.e., more in-person meetings, memos, and webinars) for implementation 
planning. Because the SSI Project requires significant school organizational 
change and commitment, signed memorandums of understanding were re¬ 
quired to secure partnerships prior to submission of the TIF grant applica¬ 
tion. 

During Year 1 (entiy and contracting), numerous consultation meetings 
were conducted to present and discuss SSI Project Absolute Priorities (HCMS 
and EES) and overall goals. Consultation meetings were scheduled with 
the following parties: (a) district administrative leaders, (b) school princi¬ 
pals, (c) district-wide committees (e.g., DEACs, SIPs). Consultation meetings 
were scheduled according to the organizational hierarchy to obtain input 
and permissions to proceed with collaborative planning. During Year 1, 
numerous changes in statewide policies on educator evaluation emerged 
that informed the planning process. For example, Student Growth Objectives 
(SGOs) were mandated by the state during Year 1, requiring immediate 
discussion and adoption into the EES. SGOs require teachers to identify 
and in many cases develop measureable learning benchmarks for student 
achievement. SGOs are to be reviewed and approved by principals. This 
state and national requirement for teacher evaluation inspired the SSI Project 
team to establish SGO consortium meetings across the four school districts to 
discuss the requirement, share current practices, and forge a framework for 
implementation across school districts. As a result of Year 1 collaboration, 
the SSI Project partners developed a two-part training series for principals 
and teachers. The first series provides principals and teachers an overview of 
the SGO process and the role that SGOs play in the EES. The second series 
provides training separately to principals and teachers. Principals are trained 
on the specific indicators to seek when approving an SGO and questions 
to ask when conferencing with teachers about the SGO for their students. 
Teachers are trained on a five-step process to design and implement measur¬ 
able SGOs. Based on consultation with district-level teacher committees on 
the EES, the SSI Project team developed a new PD designed to support 
teachers in integrating data inputs from the EES to enhance data-based 
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decision making for instructional improvements. The EES multiple measures 
approach provides teachers unique access to data streams on curriculum, 
instruction, and assessment. 

Considerable efforts were made to align project goals with state policy 
and school district initiatives to reduce perception that the project would 
cause extra, redundant effort. District administrative leaders directly commu¬ 
nicated and collaborated with their local teacher and principal unions and 
boards of educations during Year 1. Information on these communications 
was to be shared with the SSI Project team and used for planning. Unfortu¬ 
nately, school district leaders’ direct communications and collaborations with 
local principal and teacher unions on the SSI Project did not work well. The 
SSI Project team discovered that information on the SSI Project priorities and 
goals were inadequately communicated to teacher union representatives. For 
some of the school districts, school district leaders and teacher unions had 
negative collaboration histories and for the SSI Project this trend continued. 
Communication issues in some cases increased fear and frustration felt by 
teachers toward the project. Likewise, district-level communication issues 
also resulted in state-level teacher union concerns. As a result, the SSI Project 
team was required to communicate and resolve concerns related to collective 
bargaining agreements at the local and state teacher union levels. 

Based on the consultation process, the SSI Project team established 
contracts that defined roles and responsibilities, outlined EES training for 
principals and teachers, and identified PD plans for each school district for 
Year 2. Teacher training on the EES was set for May and June, and principal 
training on the EES was set for July and August in Year 1. 

During Year 1, baseline HCMS information was obtained from each 
school district, reflecting that the SSI Project team transitioned from con¬ 
tracting to information gathering according to Kurpuis (1993). Consultation 
meetings were scheduled with school district leaders and directors of human 
resources to discuss the specific components of HCMS and the availability 
of existing data at each school district. As expected, the availability of HCMS 
information varied by school districts. Examples of data collected for 2011- 
2012 and 2012-2013 (Year 1) included (a) the number of openings for 
teaching positions, (b) the number of applications received for the teacher 
openings, (c) the number of new hires that remained teaching in the school 
district, (d) the number of teachers rated as Effective and Highly Effective, 
(e) the number of teachers that remained in the school district that were 
rated as Effective and Highly Effective, and (f) the number of teachers rated 
as Effective or Highly Effective that remained in their assignments. The 
same information was collected for principals. Also, efforts were made to 
review and discuss current employee marketing, hiring, promotion, and 
dismissal material and policies. The baseline data and discussion at the school 
district level reflected limited and inconsistent HCMS data collection, which 
generated initial plans for HCMS improvements. 
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Initial meetings with directors of human resources highlighted the need 
for further training on HCMS principles and strategies. As a result, the SSI 
Project partners developed a two-track PD system focused on (a) school 
management, and (b) instructional leadership aligned with HCMS princi¬ 
ples and Interstate School Leaders Licensure Consortium (ISLLC) Standards. 
Track 1 School Management targets the evidence-based leadership practices 
that include organizational management, school climate, data-based decision 
making, and teacher evaluation. Track 2 Instructional Leadership targets 
evidence-based instructional practices that maximize teacher effectiveness, 
teacher feedback/coaching, student growth in achievement, and principal 
PD (goal setting). 

To further obtain input from principals and teachers and inform the SSI 
Project implementation planning, an online (mixed method) survey is being 
conducted to assess school administrators’ and teachers’ experiences with 
teacher evaluation during the 2012-2013 school year (SSI Project Year 1). 
The survey includes three scales, the Administrator and Teacher Forms of 
the Teacher Evaluation Experience Scale (TEES; Reddy et al., under review), 
and the Collective Efficacy Scale (CE-Scale; Goddard, Hoy & Woolfolk, 2000). 
The TEES Administrator and Teacher Forms include 39 items assessing princi¬ 
pals’ and teachers’ experiences with four factors (subscales): (1) the teacher 
evaluation system (e.g., “The evaluation system was useful”), (2) evalua¬ 
tion feedback (e.g., “My evaluation feedback was specific”), (3) evaluation 
processes (e.g., “During my feedback meeting, I was encouraged to share 
my thoughts”), and (4) motivation to change (e.g., “The teacher evaluation 
system increased my motivation to improve my classroom practice”). Items 
are rated on a 5-point Likert scale, ranging from 1 (strongly disagree ) to 
5 (strongly agree). Open-ended questions were included to further assess 
evaluation feedback and processes. Confirmatory factor analyses of input 
from 583 teachers were carried out on the four-factor (i.e., 4 subscales), 
one-factor (total scale), and five-factor models (i.e., total and 4 subscales). 
Models yielded good fit to the data based on y^/df, root mean square error of 
approximation, adjusted goodness of fit index, and the goodness of fit index. 
Information theory-based indices of relative fit (i.e., AICo, BCCo, and BICo) 
indicated that the four factor model yielded superior fit to alternative models. 
The TEES has good internal consistency, construct validity, and reliability 
and concurrent and divergent validity with CE-Scale (Reddy et al., under 
review). 

The CE-Scale (Goddard, Hoy, & Woolfolk, 2000), is a 21-item measure 
of teacher collective efficacy in schools. Items are rated on a 6-point Likert 
scale from 1 (strongly disagree) to 6 (strongly agree). The CE-Scale has 
adequate to good internal consistency, as well as construct and criterion- 
related validity (e.g., Goddard, Hoy, & Woolfolk, 2000; Hoy & Kupersmith, 
1985; Hoy & Sabo, 1998). It is anticipated that the three measures will 
generate valuable information on principals’ and teachers’ experiences with 
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educator evaluation that will inform the Year 2 implementation of the EES 
and PD opportunities. 


RECOMMENDATIONS FOR FUTURE SCHOOL REFORM 

Based on the Year 1 consultation process, we offer the following recom¬ 
mendations for future school reform initiatives. First, effective school reform 
is predicated on developing strong collaborative partnerships with key ad¬ 
ministrative leaders in schools. Consultants that forge positive and valued 
relationships with administrative leaders can guide system change, as well as 
adopt and implement innovation measurement and PD models. Consultants 
can assume new leadership roles that help administrative leaders in decision 
making and empowering other change agents (e.g., directors of curriculum, 
principals, teacher leaders) and stakeholders to aspire toward system-wide 
instructional improvement. Also, consultants can help administrative leaders 
further develop effective collaboration, communication, and leadership skills 
with school personnel. Other features of effective leadership that may be 
targeted include communication, support of self-improvement for all school 
personnel, and establishment of a climate of respect, trust, and fairness. These 
features were critically important for the SSI Project team during Year 1. The 
need to maintain direct ongoing communications with teacher representatives 
on the project was a valuable lesson learned. Other important lessons learned 
by the SSI Project team were to help school partners maintain perspective 
and flexibility to statewide and local policy changes that impact school re¬ 
form planning and implementation. For the SSI Project, successful admin¬ 
istrative leaders skillfully analyze, prioritize, and communicate essential el¬ 
ements of statewide policy into actionable, practical plans for school staff. 

Second, consultation is enhanced by adopting a multidimensional as¬ 
sessment of the cultural contexts of school systems during the development, 
adoption, and implementation of school reform. School districts include 
several cultural systems, which have unique inputs (administrators, teachers, 
students) and outputs (professional growth, safety, social climate) important 
to the consultation process. Ingraham (2009) has advocated for consultants to 
adopt a multicultural and interdisciplinary perspective of working in schools 
with particular attention to the cultural context(s) of the clients. Although the 
SSI Project includes four high-poverty school districts, each school district 
has different organizational structures and cultures. Likewise, SSI Project 
partner schools are located in diverse and unique communities (e.g., urban, 
suburban). For school reform, it is advantageous that the consultants assess 
individual and group frame of references of the school reform model and 
the potential impact on collaborative planning and implementation. As a 
result, consultants are encouraged to culturally validate reform components 
and obtain buy-in prior to implementation. 
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Third, school reform often requires changes in the roles and functions of 
school personnel (i.e., human capital management). Thus, it is important for 
consultants to be knowledgeable of principal and teacher union collective 
bargaining agreements, and be skillful in negotiating possible labor-related 
issues that may arise in work scope changes. The SSI Project team had limited 
experience in teacher union collective bargaining agreements prior to Year 
1. This is an area that warrants specialized knowledge and consultation with 
legal counsel. To avoid miscommunication among school and district leaders, 
teacher union representatives, and project-based leaders, personal meetings 
among key decision makers are preferable to indirect communication. The 
complexities of an HCMS combined with the high-stakes of an EES are 
difficult to accurately relay indirectly (e.g., district staff to teacher union 
representatives). 

Fourth, the literature on implementation science provides valuable in¬ 
formation for planning and implementing school reform. SSI Project school 
reform includes complex implementation processes. Some key elements 
include fidelity (i.e., adherence to the measurement system of PD), quality of 
delivery (i.e., skill with which the implementer delivers information/materials 
and interacts with school personnel), adaptation in delivery (i.e., changes 
made to the program, materials added or removed), and school personnel be¬ 
haviors/responsiveness (i.e., implementers and non-implementers). Berkel, 
Mauricio, Schoenfelder, and Sandler’s (2011) theoretical model of prevention 
implementation outlines behaviors of program implementers (i.e., fidelity, 
quality of delivery, adaptation) and behaviors of participants as potential me¬ 
diators or moderators of prevention outcomes. They contend that participant 
responsiveness (e.g., attendance, enthusiasm for and active participation in 
model delivery) and implementer adaptation are critical for outcome success. 
Research has shown that school personnel who adapt or refine models to 
meet context needs are likely to adopt and sustain the use of new models 
(e.g., Greenhalgh, Robert, McFarkan, Bate, & Kyriakidou, 2004; Roger, 1995). 
Guided by this literature, the SSI Project team carefully considered how 
principal and teacher trainings and support can be tailored to best fit in 
school contexts while maintaining acceptable levels of fidelity. Year 2 will 
provide valuable information on the measurement of adaptation and inform 
future implementation and research in the project. Moreover, the design of 
the EES is such that implementation fidelity of certain measures (e.g., CSS, 
MyiLOGS) directly impacts the extent to which their use contributes to the 
overall effectiveness determination. 

While implementation fidelity has been recognized as a cornerstone for 
successful school intervention, consultation, and programming (e.g., Durlack 
& Dupre, 2008; Schulte, Easton, & Parker, 2009), assessment approaches 
of school-wide implementation fidelity are lacking. This void offers oppor¬ 
tunities for measurement development, which may lead to better school 
reform implementation and success. We advocate for a multidimensional 
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measurement of school-wide implementation fidelity as shown on Table 1. 
Likewise, the measurement of adaptations in delivery is warranted. Assess¬ 
ment of implementation adaptation may uncover contextual needs and lead 
to improved model development, outcomes, and sustained use. Thus, a 
balance of implementation fidelity and adaptation in relation to outcome 
effectiveness is needed for practice and research. 

Finally, school reform would benefit from the work on innovation diffu¬ 
sion (Roger, 2003). Roger’s (2003) seminal work on innovation development, 
adoption, and implementation outlines five characteristics: adoption (i.e., 
school decision to use a new program or system), relative advantage (i.e., 
the perceived advantage of the program over what is currently used), com¬ 
patibility (i.e., program compatibility with the current state of the school), 
complexity (i.e., the perceived ease to understand or use the program), 
trialability (i.e., degree to which the program can be used on a limited 
trial basis), and observability (i.e., the degree the system or individuals 
can see the results of the program). Additional variables that have been 
found to explain variance in school-wide adoption of innovation have been 
risk (i.e., a model perceived as more risky is less likely to be adopted) 
and task issues (i.e., a model perceived as more relevant to the school 
personnel work and to improved task performance is more likely to be 
adopted; Greenhalgh et al., 2004). During Year 1, considerable effort was 
made to reduce perceived risk (variable high for educator evaluation) and 
increase SSI Project alignment with school personnel work, improved pro¬ 
fessional performance, and statewide policy. It is important to note that 
a school district’s decision to adopt a school reform model is not suffi¬ 
cient for successful implementation. Based on research related to concerns- 
based adoption models (Hall & Hord, 1987, 2001), the SSI Project team 
employed several methods to enhance adoption and implementation in 
Year 2. These methods included ongoing information on (a) objectives the 
project will accomplish for schools, (b) ways schools can use the project 
components in Year 2, (c) ways the project will impact school personnel, 
and (d) ways the project components fit in existing educational practices 
within schools. To further enhance implementation success during Year 2, 
all principals and teachers will have continuous access to information on 
project components (training materials and classroom videos) through the 
SSI Project website (www.SSIproject.rutgers.edu) and school-based training, 
coaching, and supports. Additionally, SSI Project staff will monitor EES and 
PD implementation and provide tailored supports to school personnel dur¬ 
ing Project Years 2 through 5. As needed, SSI Project leadership teachers 
and principals will discuss with school personnel the ways that project 
components may be adapted with integrity to maintain effectiveness. For 
example, educators receive PD to support their ability to integrate EES data 
related to curriculum, instruction, and assessment, and use them to make 
data-based decisions for instructional improvement. It is anticipated that 
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implementation training and supports will be further refined as the project 
proceeds. 


CONCLUSIONS 

In summary, the success of the SSI Project relies on interdisciplinary col¬ 
laboration and input across school systems, in order to implement a fair 
and balanced EES based on the principles of school psychology. Educators 
in a variety of roles offer unique and complementary skills that forge an 
effective leadership team for enhancing school capacity for student growth 
in learning. System consultation can be used to efficiently and effectively 
guide leadership teams in designing and implementing innovative school¬ 
wide reform models. As noted, successful school reform is predicated on 
school systems defining, identifying, developing, and rewarding educator 
effectiveness (Odden, 2011). Human capital decisions that are informed 
by rigorous multidimensional assessment and targeted evidence-based PD 
(emphasizing improvement for all ) are critical for the next generation of 
school-wide reform initiatives. 
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